AD-757  693 


THE  USE  OF  CONFIDENTIAL  INSTRUCTOR 
RATINGS  FOR  THE  PREDICTION  OF  SUCCESS 
IN  NAVAL  UNDERGRADUATE  PILOT  TRAINING 

Wayne  L.  Waag,  et  al 


Naval  Aerospace  Medical  Research  Laboratory 
Pensacola,  Florida 

7  February  1973 


**■ 


DISTRIBUTED  BY: 


National  Technical  Information  Service 
U.  S.  DEPARTMENT  OF  COMMERCE 

5285  Port  Royal  Road,  Springfield  Va.  22151 


i 


AD757693 


NAMRl  -  ins 


THE  USE  OF  CONFIDENTIAL  INSTRUCTOR  RATINGS  FOR  THE  PREDICTION  OF 
SUCCESS  IN  NAVAL  UNDERGRADUATE  PILOT  TRAINING 

Wayne  L.  Waag,  LT  Richard  H.  Shannon,  MSC,  USN,  and  Rosalie  K.  Ambler 


u*wsa&®3r 


7  February  1973 


Approved  for  public  release;  distribution  unlimited. 


Unclassified 


DOCUMENT  CONTROL  DATA  •  R  4  D 

{Security  classillcatton  ol  title,  body  of  obstruct  and  tndcxind  annotation  must  be  entered  »>im  the  overall  report  in  classified) 


Unclassified 


I  ORIGINATING  AC  TIVIT.Y  Tfofpo/AfA  anlhofi  la*.  REPORT  SECURITY  CLASSIFICATION 

Naval  Aerospace  Medical  Research  Laboratory 
Naval  Aerospace  Medical  Institute 
Naval  Aerospace  and  Regional  Medical  Center 

Pensacola,  Florida  32512  _  _ 


3.  REPORT  TITLE 

THE  USE  OF  CONFIDENTIAL  INSTRUCTOR  RATINGS  FOR  THE  PREDICTION  OF  SUCCESS  IN  NAVAL  UNDER- 
GRADUATE  PILOT  TRAINING. 


4.  descriptive  NOTES  (Type  ol  report  end  inclusive  dates) 


5.  autmor(S)  (First  name,  middle  initial,  tail  nimr) 

Wayne  L.  Waag,  Ph.D., 

Richard  H.  Shannon,  LT  MSC  USN, 
and  Roaolie  K.  Ambler,  M.S. 


t  REPOR  T  DA  TE 


7  February  1973 


•«.  CONTRACT  or  grant  no 


7 a.  TOTAL  no  OF  PAGES  7 b.  NO-  OF  REFS 


9m.  ORIGINATOR'S  REPORT  NUMBER(S) 


6.  projec  T  NO  MF51 . 524 . 002 . 5013  .DX5X 


NAMRL  -  1175 


10.  DISTRIBUTION  STATEMENT 


Approved  for  public  release;  distribution  unlimited. 


9b.  other  REPORT  NO(S)  ('Any  other  numbers  that  may  be  as  signed 
this  report) 


12-  SPONSORING  M1LI  TARY  ACTIVITY 


13.  ABSTRACT 


Previous  investigations  have  reported  significant  relationships  between  confidential  instructor  ratings  in  early  primary 
e  and  later  success  in  Naval  flight  training.  Such  ratings  were  found  to  increase  significantly  the  validities  derived  solely 


phase  and  later  success  in  Naval  flight  training.  Such  ratings  were  found  to  increase  significantly  the  validities  derived  solely 
from  selection  test  scores.  However,  such  findings  do  not  guarantee  that  confidential  ratings  would  augment  the  validities 
derived  from  the  combined  array  of  selection  and  ear  ly  training  variables  which  are  used  in  the  current  Student  Pilot  Prediction 
System.  The  purpose  of  the  present  study  was  to  determine  whether  such  confidential  ratings  provided  non-redundant  informa¬ 
tion  which  would  increase  the  predictive  value  of  the  present  system. 

The  results  clearly  indicated  that  confidential  ratings  obtained  from  Primary  flight  instructors  provided  information 
relating  to  the  student's  probability  of  receiving  his  wings.  Such  ratings  were  found  to  significantly  increase  the  predictive 
validities  derived  from  the  variables  which  are  used  currently  in  the  Student  Pilot  Prediction  System.  Such  findings  suggest 
that  these  confidential  evaluations  provide  additional  information  beyond  that  which  is  reflected  in  the  grades  he  assigns. 

It  is  recommended  that  confidential  instructor  ratings  be  implemented  on  a  permanent  basis  in  the  presolo  stage.  The 
present  Student  Pilot  Prediction  System  should  be  revised  to  incorporate  this  information. 


,1473  (PAGE  n 


S/N  0101.807.6801 


Unclassified 

Security  Classification 


_ Undggfoj _ 

Security  CI»»»l(lc«Uon 

« 

K  C  V  MO  RD1 

Not  Perform  mi  rr  Mt—rnaft 
laatmctor  R»ti«p 
Aviation  Traiainf 
Secondary  Selection 


DD  .'.r..1473  ib«c«, 

(PAGE  2) 


SUMMARY  PAGE 


PROBLEM 


Previous  investigations  have  reported  significant  relationships  between 
confidential  instructor  ratings  in  early  primary  phase  and  later  success  in  Naval 
flight  training.  Such  ratings  were  found  to  increase  significantly  the  validities 
derived  solely  from  selection  test  scores.  However,  such  findings  do  not 
guarantee  that  confidential  ratings  would  augment  the  validities  derived  from 
the  combined  array  of  selection  and  early  training  variables  which  are  used  in 
the  current  Student  Pilot  Prediction  System .  The  purpose  of  the  present  study 
was  to  determine  whether  such  confidential  ratings  provided  non-redundant 
information  which  would  increase  the  predictive  value  of  the  present  system  . 


FINDINGS 


The  results  clearly  indicated  that  confidential  ratings  obtained  from 
Primary  flight  instructors  provided  information  relating  to  the  student's  proba¬ 
bility  of  receiving  his  wings.  Such  ratings  were  found  to  significantly  increase 
thb  predictive  validities  derived  from  the  variables  which  are  used  currently  in 
the  Student  Pilot  Prediction  System .  Such  findings  suggest  that  these  confi¬ 
dential  evaluations  provide  additional  information  beyond  that  which  is 
reflected  in  the  grades  he  assigns . 


RECOMMENDATIONS 


It  is  recommended  that  confidential  instructor  ratings  be  implemented  on 
a  permanent  basis  in  the  presolo  stage .  The  present  Student  Pilot  Prediction 
System  should  be  revised  to  incorporate  this  information . 
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INTRODUCTION 


The  flight  instructor  is  required  to  serve  a  dual  function.  Although  his 
principle  duty  is  to  teach  students  to  fly ,  he  must  aiso  evaluate  their  progress 
for  the  record.  Such  evaluations  are  reflected  in  grades  which  become  a  per¬ 
manent  part  of  the  student's  flight  jacket  and  are  subject  to  the  scrutiny  of  both 
the  training  command  and  the  student.  As  a  result  of  grading  standards  set  by 
the  training  command  and  subtle  pressures  invoked  in  face-to-face  evalua¬ 
tions,  it  is  possible  that  an  instructor's  actual  opinion  regarding  a  student's 
progress  may  not  be  completely  reflected  in  the  grades  he  assigns.  Conse¬ 
quently,  the  utilization  of  confidential  instructor  opinions  may  add  significantly 
to  the  information  available  trom  assigned  grades . 


In  an  early  study,  Martoccia  and  Nelson  (4)  compared  actual  flight 
grades  and  expressed  instructor  ratings  as  predictors  of  later  success  in  Basic 
training .  For  one  item  concerning  the  likelihood  of  the  student  receiving  his 
wings,  correlations  ranging  trom  .26  to  .33  were  reported.  For  the  actual 
grades,  the  reported  r  was  only  .17.  In  a  more  recent  study,  Berkshire  and 
Ambler  (2)  reported  an  r  of  .35  between  instructor  predictions  and  attrition. 
Such  ratings  were  obtained  for  a  sample  of  students  receiving  a  series  of  indoc¬ 
trination  flights  prior  to  their  academic  training .  The  addition  of  these 
instructor  opinions  were  found  to  increase  significantly  the  predictive  validities 
derived  from  the  Aviation  Qualirication  Test  (AQT)  and  the  Flight  Aptitude  Rat¬ 
ing  (FAR).  The  multiple  R  was  increased  from  .239  to  .388.  Such  findings 
indicate  that  instructor  ratings  significantly  enhance  the  predictive  validities 
derived  solely  from  selection  test  scores . 


The  present  Student  Pilot  Prediction  System  considers  more  than  just 
selection  test  scores .  Predictions  derived  at  a  given  stage  of  training  depend 
upon  a  number  of  indices  including  previous  flight  grades,  academic  grades, 
military  grades ,  and  exemption  examinations ,  as  well  as  the  selection  test 
scores  (3) .  If  instructor  ratings  are  to  yield  useful  information,  it  must  be 
demonstrated  that  they  significantly  increase  the  validities  of  the  current  pre¬ 
diction  system;  that  is ,  the  addition  of  such  confidential  ratings  must  explain  a 
significant  non-redundant  amount  of  the  criterion  variance.  The  purpose  of 
the  present  investigation  was  to  determine  if,  in  fact,  the  use  of  instructor  rat¬ 
ings  would  increase  significantly  the  validity  of  the  present  Student  Pilot  Pre¬ 
diction  System . 
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METHOD 


At  present ,  different  sets  of  equations  have  been  developed  for  flight 
students  entering  from  Aviation  Officer  Candidate  (AOC)  sources  and  Officer 
Under  Instruction  (01)  sources.  AOC s  are  procured  from  the  civilian  popula¬ 
tion  and  most  have  no  prior  experience  with  the  military.  OIs,  on  the  other 
tiand,  have  obtained  their  commission  elsewhere  before  entering  the  flight  pro¬ 
gram.  AOCs  must  successfully  complete  Aviation  Officer  Candidate  School 
before  entering  ground  school  training  (Environmental  Indoctrination).  OIs, 
however ,  enter  directly  into  ground  school .  Upon  completion  of  ground  school 
training ,  students  are  sent  to  Primary  flight  training  which  consists  of  two 
stages ,  Pre-Solo  and  Precision . 


The  Pre-Solo  stage  of  training  for  most  students  is  the  first  encounter 
with  flying  an  aircraft.  It  is  during  this  period  that  the  student  is  assigned  to  a 
single  instructor  except  tor  off-wing  and  check  hops.  Consequently,  the 
instructor  is  in  the  unique  position  of  observing  the  student's  initial  reactions 
to  flight  as  well  as  the  initial  progress  he  makes.  In  other  words,  the  role  of 
the  primary  flight  instructor  should  be  an  excellent  vantage  point  for  evaluat¬ 
ing  the  potential  success  of  his  students . 


Confidential  instructor  ratings  of  student  pilot  performance  were 
obtained  for  a  sample  of  1276  student  aviators  completing  Primary  Flight  Train¬ 
ing  between  July  1969  and  December  1970.  Specifically,  instructors  were  asked 
after  the  7th  or  8th  hop  to  rate  their  students  on  each  of  four  questions  concern¬ 
ing:  1)  the  probability  of  the  student  obtaining  his  wings;  2)  the  student's 
motivation;  3)-  the  student's  headwork;  and  4)  the  student's  reaction  to  stress . 
The  complete  questionnaire  is  presented  in  Appendix  A . 


For  each  student,  the  following  information  was  obtained:  1)  Selection 
test  scores  for  the  Aviation  Qualification  Test,  Mechanical  Comprehension  Test, 
Spatial  Apperception  Test,  and  Biographical  Inventory.  All  students  must 
qualify  on  these  tests  before  receiving  orders  to  flight  training  (1) ;  2)  Mathe¬ 

matics  and  Physics  Exemption  test  scores  administered  at  the  beginning  of  aca¬ 
demic  training;  3)  Environmental  Indoctrination  grades  including  Aviation  Phy¬ 
siology,  Aerodynamics.,  and  Engineering;  4)  Confidential  instructor  ratings  on 
the  four  items  of  the  questionnaire;  and  5)  the  Pre-Solo  grade.  Furthermore, 
each  student  was  categorized  according  to:  1)  Procurement  source — AOC  vs  OI; 

2)  Branch  of  Service-Marine  vs  Navy;  and  3j  Criterion  of  Success-com¬ 
pletion  vs  attrition.  For  all  AOC  students,  two  additional  measures  were 
obtained ,  the  Officer-Like  Qualities  (OLQ)  rating ,  and  the  Peer  Rating . 
Instructors  providing  the  confidential  ratings  were  categorized  according  to:  1) 


2 


Service — Marine  vs  Navy;  and  2)  Experience — SERGRAD1  vs  fleet-exper¬ 
ienced.  According  to  these  different  categories,  the  total  sample  of  1276  was 
broken  down  as  follows:  1)  460  AOCs  and  816  OIs;  2)  216  Marine  and  1060 
Navy;  3)  967  completions  and  309  attritions;  4)  148  taught  by  Marine  instruc¬ 
tors  and  1128  taught  by  Navy  instructors;  and  5)  327  taught  by  fleet-exper¬ 
ienced  pilots  and  949  taught  by  SERGRADs. 


RESULTS 


Table  I  presents  a  comparison  between  AOCs  and  OIs  for  each  of  the 
obtained  measures.  For  most  measures  yielding  a  significant  difference,  the 
01  group  produced  better  scores.  The  only  exception  was  the  BI .  This  differ¬ 
ence  was  due  to  the  fact  that  BI  selection  standards  for  AOCs  tended  at  that  time 
to  be  higher  than  for  OIs.  An  intercorrelation  matrix  was  then  obtained  for 
each  sample.  These  are  presented  in  Appendix  B  .  The  Pearson  zero-order 
correlations  between  each  measure  and  pass-attrite  are  presented  in  Table  II . 

It  is  interesting  to  note  the  differences  across  the  two  samples .  From  these 
correlations ,  it  was  apparent  that  the  prediction  equations  developed  for  each 
sample  would  emphasize  different  measures . 


Three  series  of  regression  analyses  were  then  performed  for  each 
sample  using  a  forward  selection  procedure .  For  Analyses  I ,  the  set  of 
potential  predictors  included  all  available  information  up  to  but  not  including 
Primary  flight  training .  Weight  reversals  were  suppressed  so  that  a  variable 
was  eliminated  if  the  sign  of  its  beta  weight  did  not  coincide  with  the  sign  of  its 
zero-order  correlation  with  the  criterion .  These  statistical  decisions  were  con¬ 
sistent  with  the  procedures  used  to  develop  the  current  set  of  equations  for  the 
Student  Prediction  System  (3) . 


In  the  second  set  of  analyses,  or  Analyses  II,  those  variables  selected 
during  Analyses  I  were  first  forced  into  the  regression  equation.  Primary 
flignt  training  variables  were  then  entered  into  the  equation  on  a  sequential 
basis;  that  is,  in  a  sequence  which  simulated  the  actual  availability  of  informa¬ 
tion.  Consequently,  instructor  type  information  was  entered  first,  followed  by 
the  confidential  ratings,  and  finally  by  the  Pre-Solo  grade.  Again  weight 
reversals  were  suppressed  and  variables  with  an  F-ratio  less  than  one  were 
excluded . 


1SERGRAD  is  an  acronym  for  Selected  Returned  Graduates  whicn  identifies 
instructors  who  are  assigned  to  instructor  duty  immediately  upon  completion  of 
undergraduate  pilot  training . 
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Table  I 


Comparison  of  Performance  Measures  between  AOCs  and  OIs 


Performance 

Measure 

AOC 

Means 

OI 

z 

Value 

Student:  Marine  vs  Navy 

_____ 

.314 

-  -  + 

Aviation  Qualification  Test 

84.586 

86.945 

3.726** 

Mechanical  Comprehension  Test 

60.379 

60.098 

.643 

spatial  Apperception  Test 

21.654 

21.706 

.173 

Biographical  Inventory 

43.396 

40.074 

4.303** 

Math  Exemption 

38.396 

47.039 

12.617** 

Physics  Exemption 

38.932 

48.787 

13.660** 

Aviation  Physiology 

50.963 

52.038 

2.161* 

Aerodynamics 

52.094 

53.150 

1.878 

Engineering 

51.612 

51.777 

.295 

Officer-Like  Qualities 

51.392 

-  -  - 

-  -  - 

Peer  Rating 

51.118 

-  -  - 

-  -  - 

Instructor:  Marine  vs  Navy 

.102 

.124 

1.185+ 

Instructor:  1st  vs  2nd  Tour 

.254 

.257 

.122+ 

Item  1-Wings 

8.589 

9.208 

3.842** 

Item  1-Motivation 

9.111 

9.658 

3.681** 

Item  3-Headwork 

7.763 

8.311 

3.351** 

Item  4-Stress 

7.820 

8.516 

4.358** 

Pre-Solo  Grade 

3.015 

3.027 

3.243** 

Pass-Attrite 

.630 

.830 

7.645** 

**  p  <  .01 

*  p  <  .05 

+  These  are  percentages  resulting  from  coding  the  1st  category  1,  and  the 
2nd  category  0. 
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Table  II 


Zero-Order  Correlations  Between  Performance  Measures  and 

Pass-Attrite 


Performance 

Measure 

AOC 

Sample 

OI 

Sample 

Student:  Marine  vs  Navy 

_  — 

.145** 

Aviation  Qualification  Test 

.096* 

.039 

Mechanical  Comprehension  Test 

.146** 

.030 

Spatial  Apperception  Test 

.025 

.108** 

Biographical  Inventory 

.039 

.097** 

Math  Exemption 

.123** 

.052 

Physics  Exemption 

.152** 

.025 

Aviation  Physiology 

.125** 

.075* 

Aerodynamics 

.156** 

.095** 

Engineering 

.165** 

.090** 

Officer-Like  Qualities 

.128** 

-  - 

Peer  Rating 

.100* 

-  - 

Instructor:  Marine  vs  Navy 

-.009 

.081* 

Instructor:  1st  vs  2nd  Tour 

-.049 

.043 

Item  1— Wings 

.265** 

.166** 

Item  2— Motivation 

.146** 

.149** 

Item  3 — Headwork 

.212** 

.137** 

Item  4 — Stress 

.202** 

.163** 

Pre-Solo  Grade 

.307** 

.201** 

**  p  <  .01 

*  p  <  .05 
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Table  III  presents  the  results  of  Analysis  n  for  the  AOC  sample.  As 
indicated,  5.7%  of  the  criterion  variance  could  be  explained  by  information 
available  prior  to  Primary  flight  training .  The  addition  of  the  instructor  rating 
on  item  1  increased  the  explained  variance  by  4.6  percentage  points,  a  fairly 
substantial  amount.  Finally,  upon  entering  the  Pre-Solo  grade,  an  additional 
3.3%  oi  the  criterion  variance  was  explained.  The  final  equation  yielded  a 
multiple  R  of  .  369 . 


Table  III 

Summary  of  Regression  Analysis  II  for  AOC  Sample 


Variable 

Entered 

Multiple  Multiple 
R  R2 

Increase  In 
R2 

F -Ratio  For 
Inclusion 

Engineering 

.165 

.027 

.027 

12.834 

Officer-Like  Qualities 

.204 

.041 

.014 

6.869 

Physics  Exemption 

.223 

.050 

.009 

3.858 

Aerodynamics 

.233 

.055 

.005 

2.314 

Mechanical  Comprehension 
Test 

.238 

.057 

.0u2 

1.128 

Item  1-Wings 

.321 

.103 

.046 

23.230 

Pre-Solo  Grade 

.369 

.136 

.033 

17.450 

Table  IV  presents  the  results  of  Analysis  n  for  the  01  sample .  As  indi¬ 
cated  ,  6 . 0%  of  the  criterion  variance  could  be  explained  by  information  available 
prior  to  Primary  flight  training .  The  addition  of  instructor  information,  includ¬ 
ing  the  Marine-Navy  dichotomy  and  the  ratings  on  three  of  the  four  items,  added 
3,1%  to  the  explained  variance.  After  entering  the  Pre-Solo  grade,  only  1.0% 
additional  criterion  variance  was  explained .  The  final  equation  yielded  a 
multiple  R  of  .318. 


While  these  findings  indicate  that  instructor  ratings  significantly 
increase  the  validities  of  equations  confined  to  "early"  predictors,  the  possi¬ 
bility  remained  that  such  increases  may  have  been  negligible  had  the  Pre-Solo 
grade  been  entered  first.  To  test  this  possibility  an  additional  set  of  analyses 
or  Analyses  III  was  performed .  All  information  other  than  the  instructor  data 
was  initially  forced  into  the  equation  regardless  of  the  value  of  the  F-ratio  or 
sign  agreement  between  the  beta  weights  and  zero-order  correlations.  In  this 
manner ,  the  maximum  multiple  R  is  obtained .  For  the  AOC  sample ,  the  obtained 
R  was  .367,  which  explained  13.5%  of  the  criterion  variance.  Instructor  rating 
data  was  then  forced  into  the  regression  equation ,  and  the  multiple  R  was 


6 


Table  IV 


Summary  of  Regression  Analysis  II  for  AOC  Sample 


Variable  Multiple  Multiple  Increase  in  F-Ratio  For 


Entered 

R 

R2 

R2 

Inclusion 

Student:  Marine  vs  Navy 

.145 

.021 

.021 

17.428 

Aerodynamics 

.205 

.042 

•  U21 

17.897 

Spatial  Apperception  Test 

.222 

.049 

.007 

6.338 

Biographical'  Inventory 

.235 

.U55 

.006 

4.947 

Aviation  Physiology 

.242 

.059 

.004 

2.840 

Engineering 

.245 

.OBO 

,00x 

1.237 

instructor:  Marine  vs  Navy 

.252 

.063 

.003 

2.843 

Item  i-Wings 

,296 

.087 

.024 

21.359 

Item  2-Motivation 

.299 

.089 

.002 

1.632 

Item  4-Stress 

.302 

.091 

.002 

1.913 

Pre-Solo  Grade 

.318 

.101 

.010 

8.543 

increased  to  .387,  which  explained 

15.0%  of  the  variance. 

The  addition  of  th 

instructor  information,  therefore,  explained  an  additional  1.5%  ot  the  total 
variance . 


For  the  01  sample,  the  obtained  R  using  all  predictors  other  than 
instructor  data  was  .302,  which  explained  9.1%  of  the  criterion  variance.  The 
addition  of  the  instructor  information  increased  the  R  to  .330,  which  explained 
10.883%  of  the  variance.  The  addition  of  the  instructor  therefore  explained  an 
extra  1.8%  of  the  total  variance.  For  both  samples  such  increases  were  statisti¬ 
cally  significant.  A  comparison  of  the  key  increases  in  explained  criterion 
variance  after  the  introduction  of  the  instructor  ratings  are  presented  in  Table 
V  for  summary  purposes . 


DISCUSSION 


The  results  of  this  investigation  clearly  indicate  that  confidential  rat¬ 
ings  obtained  from  Primary  flight  instructors  provided  valuable  information 
which  were  related  to  the  student's  likelihood  of  receiving  his  wings.  More 
importantly,  such  ratings  significantly  increased  the  predictive  validities 
derived  from  information  which  is  currently  used  in  the  Student  Pilot  Prediction 
System .  This  indicates  that  the  instructor's  confidential  evaluations  provide 
additional  information  beyond  that  which  is  reflected  in  the  grades  he  assigns. 
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Table  V 


Comparative  Increases  in  Per  Cent  of  Criterion  Variance 
Explained  by  Introduction  of  Instructor 
Ratings 


Student 

Type 

Analyses  II* 

Analyses  III* 

AOC 

4.6% 

1.5% 

ul 

3.1% 

1.8% 

♦Attention  is  called  to  fact  that  Analyses  II  utilized  only  the  significant  variables 
from  Analyses  I  while  Analyses  III  forced  all  variables . 


It  should  be  emphasized  that  the  findings  of  this  study  are  based  upon 
attritions  occurring  after  Pre-Solo  Primary  training .  In  addition  to  the  sample 
of  1276  students  completing  Pre-Solo,  there  were  43  students  who  received  rat¬ 
ings,  but  did  not  finish  Pre-Solo.  Had  these  records  been  included  in  the 
sample,  the  validities  of  the  instructor  ratings  would  have  been  increased. 
However ,  it  was  felt  that  these  confidential  ratings  would  be  of  greater  value 
if  they  could  be  shown  to  predict  late  attritions  during  undergraduate  training . 
For  this  reason,  the  authors  feel  these  results  to  be  highly  encouraging .  Such 
data  strongly  suggest  that  the  implementation  of  instructor  ratings  would  signi¬ 
ficantly  enhance  the  validity  of  the  present  prediction  system .  The  greatest 
strength  of  these  instructor  ratings  appears  to  be  in  the  augmentation  of  the 
prediction  capability  of  later  attrition  before  the  pre-solo  stage  grade  is  earned . 
However ,  there  is  enough  statistical  evidence  in  the  conservative  approach  of 
Analysis  HI  to  indicate  significant  augmentation  of  prediction  capability  even 
after  the  pre-solo  grade  is  earned . 


Although  the  usual  replicative  cross-validation  study  has  not  yet  been 
performed ,  there  are  several  reasons  which  support  the  recommendation  that 
such  ratings  be  considered  for  implementation.  First,  the  present  results  do 
replicate  the  previous  findings  that:  (1)  instructor  ratings  are  significantly 
related  to  success  in  naval  air  training;  and  (2)  instructor  ratings  significantly 
increase  the  predictive  validities  derived  solely  from  the  selection  test  scores . 
Second,  the  finding  of  augmentative  capability  in  both  the  AOC  and  01  sample 
may  be  considered  to  be  similar  to  the  procedure  of  cross-validation  by  sample 
fractionation.  Third,  the  results  are  based  upon  a  relatively  large  sample 
size.  For  these  reasons,  consideration  for  implementation  without  the  usual 
cross-validation  procedures  is  recommended . 


8 


A  word  of  caution  seems  to  be  in  order,  however.  The  results  ot  this 
investigation  are  based  upon  data  obtained  as  part  of  an  experimental  research 
program.  The  effects  of  implementation  of  such  a  rating  form  on  a  permanent 
basis  are  unknown.  In  the  event,  however,  these  ratings  became  public,  it  is  a 
reasonable  certainty  their  usefulness  would  be  lost.  The  value  of  such  ratings 
derives  from  the  fact  ihat  it  enables  the  flight  instructor  to  express  his  true 
opinion  concerning  a  student  without  having  to  defend  his  judgment.  Umike  the 
grades  he  assigns ,  such  confidential  ratings  would  have  no  direct  consequence 
upon  the  student.  The  instructor  would  not  have  to  defend  his  evaluation  to 
either  the  student  or  the  members  of  a  Student  Pilot  Disposition  Board  (SPDB) . 
Its  value  would  result  entirely  from  the  increased  validity  of  the  student's  pre¬ 
dicted  success  or  failure  in  the  program. 
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APPENDIX  A 


INSTRUCTOR'S  RATING 


Instructor's  name _ 

Student's  name _ 

Jacket  number _ 

What  is  the  last  hop  this  student  completed?  ______________________ 

Studies  nave  shown  that  primary  flight  instructors  are  in  the  best  possible  posi¬ 
tion  to  make  an  early  evaluation  ot  an  individual  student.  Such  an  early  assess¬ 
ment  would  be  a  valuable  addition  to  the  information  administrators  now  have 
available  when  evaluating  a  student.  This  questionnaire  will  not  be  kept  in  the 
student's  jacket  but  in  a  separate  file. 

Below  are  four  questions  for  you  to  answer.  The  questions  are  subjective  and 
are  difficult  to  answer  definitely .  To  get  an  accurate  assessment  of  your 
opinion  please  check  the  line  on  the  continuum  which  best  represents  your  feel¬ 
ing. 

1)  IN  YOUR  OPINION  WILL  THIS  STUDENT  GET  HIS  WINGS? 

definite  probably  definite 

no  will  yes 

2)  HOW  WELL  MOTIVATED  IS  THIS  MAN  TO  BECOME  A  NAVAL  AVIATOR? 


not  well  extremely 

very  motivated  well 

3)  HOW  IS  THIS  STUDENT'S  HEADWORK? 


poor  good  outstanding 

headwork 

4)  HOW  MUCH  CONTROL  DOES  THIS  MAN  HAVE  WHEN  UNDER  STRESS? 


poor 


good 


outstanding 

control 
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Table  B-i 

Zero-Order  Correlations  for  AOC  Sample 
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Table  B- 2 

Zero-Order  Correlations  for  01  Sample 


